This disclosure relates generally to the field of image processing and, more particularly, to various blending techniques for use in generating wide area-of-view images.
One conventional method to generate a wide area-of-view image from a sequence of images (frames) is illustrated in FIG. 1. To begin, a sequence of frames is captured (block 100); frames 1 through 3. The frames are then registered (block 105), identifying the regions of overlap between successive frames (region 110 between frames 1 and 2, and region 115 between frames 2 and 3). Once regions 110 and 115 have been specified, a path or seam is identified through each region (block 120). Here, seam 125 through region 110 and seam 130 through region 115. In accordance with standard scene-cut algorithms, seams 125 and 130 are generally selected to pass through the most similar pixels between each image found in their respective overlap region (i.e., frames 1 and 2 in region 125 and frames 2 and 3 in region 130). As a result, seams 125 and 130 are typically placed outside of, but immediately adjacent to, moving objects within the respective overlap region. With seams 125 and 130 identified, a blend operation across each is performed (block 135); the result of which is final wide area-of-view image 140.
The role of blending operation 135 is to mask or obfuscate the differences or transition between two frames. One standard approach to do this uses a process known as “gradient domain” blending and consists of constructing the gradient field of final image 140 by copying the gradient fields of each image on the corresponding sides of the identified seam (e.g., referring to identifier 145, the gradient fields across seam 125 would be gradient field A from frame 1 and gradient field B from frame 2). Once this is done, the final image is generated by integrating over the gradients across the seam. Reconstructing a final wide angle-of-view image from its gradient fields requires substantial computational resources (to solve Poisson partial differential equations); resources that do not permit the real-time generation of wide angle-of-view images on common hand-held devices such as, for example, personal electronic devices having embedded image sensors such as mobile telephones, personal music players, tablet computer systems, and personal gaming devices. Gradient domain blending is also unable to mask registration errors visible along objects that span multiple frames.
Another approach, referred to as “image cross-fading,” masks the transition between two frames by cross-fading pixel values from each frame along the transition seam (e.g., 125 and 130). This generally consists of calculating a weighted average between the corresponding pixel values in the two frames, where the weight given to each pixel decreases smoothly while approaching the seam and vanishes at some distance after passing the seam. For example, the weight given each pixel from frame 1 in region 110 can decrease smoothly from 1 to 0 while crossing seam 125 from left to right. Similarly, the weight given each pixel from frame 2 in region 110 can decrease smoothly from 1 to 0 while crossing seam 125 from right to left. Exactly on seam 125, pixels from both frame 1 and frame 2 will have the same weight, 0.5. Reconstructing a final wide angle-of-view image using image cross-fading techniques can result in both ghosting (manifested by preserving parts of a moving object that is close to a transition seam) and banding artifacts (manifested in smooth areas in the images such as sky, constant color walls and fine textures).